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1.  Introduction 

This  paper  is  a  continuation  of  the  results  presented  in  two  earlier  papers 
[20],  [21]  and  may  be  read  as  the  sequel.  A  brief  account  of  their  results  will, 
however,  be  given  here  in  order  to  make  this  paper  selfcontained.  The  subject 
under  study  is  the  distribution  of  the  integrals  of  the  form 

(1.1)  Y(t)  =  P/(X(t),  z)dT, 

Jo 

where  X(t),  t  ^  0,  is  a  continuous  time  parameter  stochastic  process  defined  on 
a  probability  space  (£2,  &),  with  3C  as  its  state  space,  and  /  is  a  nonnegative 

(measurable)  function  defined  on  SC  x  [0,  go).  Here  it  is  assumed  that  the 
integral  Y {t)  exists  and  is  finite  almost  surely  for  every  t  >  0. 

The  integrals  Y (t)  arise  in  several  domains  of  application  such  as  in  the  theory 
of  inventories  and  storage  (see  Moran  [13],  Naddor  [14]),  and  in  the  study  of 
the  cost  of  the  flow  stopping  incident  involved  in  the  automobile  traffic  jams 
(see  Gaver  [9],  Daley  [4],  and  Daley  and  Jacbos  [5]).  Such  integrals  are  also 
encountered  in  certain  stochastic  models  suitable  for  the  study  of  response  time 
distributions  arising  in  various  live  situations  (see  Puri  [16],  [18],  [19]).  In  fact 
in  [18],  it  was  shown  that  such  a  distribution  is  equivalent  to  the  study  of  an 
integral  of  the  type  (1.1). 

In  [20],  the  work  done  by  several  authors  in  the  past  on  the  integral  functionals 
of  stochastic  processes  was  briefly  surveyed.  But  more  importantly  a  method  was 
introduced  for  obtaining  the  distribution  of  Y(t).  This  method  is  based  on  a 
“quantal  response  process”  Z(t)  defined  for  a  hypothetical  animal.  By  definition 
Z(t)  equals  one  if  the  animal  is  alive  at  time  t  and  is  equal  to  zero  otherwise.  In 
particular,  it  is  assumed  that 

(1.2)  P(Z(t  +  At  =  0| Z{t)  =  1,  X(t)  =  x)  =  Sf(x,  t)At  +  o(At), 
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with  Z( 0)  =  1  and  <5  a  nonnegative  constant.  Here  the  state  “zero”  is  an 
absorption  state  for  the  process  Z(t).  It  is  easy  to  establish  by  using  a  standard 
argument  that 


(1.3)  P{Z(t)  =  l)  =  E  ^exp  j-  S  J  f(X( t),  t)  dx 

which  in  turn  gives  the  Laplace  transform  (L.t.)  of  the  integral  Y (t).  Thus,  the 
study  of  the  distribution  of  the  integral  Y (t)  can  be  carried  out  equivalently  by 
studying  the  process  Z(t).  Note  that  the  quantal  response  process  Z(t)  does  not 
influence  the  process  X  ( t )  in  any  way,  rather,  as  is  clear  from  (1.2),  it  is  influenced 
itself  by  the  growth  of  the  process  X(t).  Again,  as  was  pointed  out  in  [20],/  is 
assumed  to  be  nonnegative  without  loss  of  generality.  Finally,  in  [20]  and  [21], 
this  method  was  applied  to  the  case  of  Markov  chains.  The  results  obtained  there 
are  summarized  in  the  next  section  for  later  use. 


2.  The  case  of  Markov  chains 

Consider  a  time  homogeneous  Markov  chain  (M.c.)  X(t)  wither  =  {1, 2,  •  •  •}, 
constructively  defined  as  follows.  If  X(tt)  =  i  at  some  epoch  tly  the  value  of 
X(t)  will  remain  constant  for  an  interval  tx  ^  t  <  tl  +  x,  whose  random 
duration  t  is  exponentially  distributed  with  density  function  ct  exp  { —  c{x} , 
Ci  ^  0;  the  probability  that  X(<x  +  t)  —  j  is  pijy  where  the  matrix  p  =  (py), 
is  a  stochastic  transition  matrix.  By  assumption,  the  quantities  c{  and  ptj  are 
independent  of  time.  Also,  we  assume  that  ct  <  oo  for  all  *  so  that  all  the  states 
of  3C  are  stable.  The  sample  paths  of  the  process  are  assumed  to  be  right  con¬ 
tinuous.  Since  the  process  is  defined  constructively,  it  is  separable.  Also,  it  is 
evident  from  the  construction  that  the  process  (X(f),  Z(t))  is  a  Markov  process 
with  state  space  9C  =  {(i,  r);i  =  1,  2,  3,  •  •  •  ;  r  =  0,  1}.  In  [20]  and  [21],  the 
above  method  was  applied  to  Markov  chains  under  the  assumption  that  / 
depends  only  on  X(t)  and  not  explicitly  on  t,  in  which  case,  in  order  to  specify/, 
we  are  given  a  sequence  of  numbers  f{i)  =  /{,  with  0  ^  /,•  <  oo,  i  =  1,  2,  •  •  • . 

Pij(t)  =  P(X(t)  =j\X(0)  =  i), 

Pij(t)  =  P{X(t)  =j,  Z(t)  =  1|X(0)  =  i,  Z( 0)  =  1); 

Ttij{oL)  =  exp  {-  a t}Pij(t)dt, 

Ttiji a)  =  exp  {-  <xt}Pij{t)  dt, 

n(  a)  =  (7c0(a)),  ?c(a)  =  (jcy(  a)); 


Let 

(2.1) 

(2.2) 
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C  =  (SijCi), 

1  =  (1,  1,1,  ), 


I  =  (S<j), 

f  =  (Sufi); 


where  i,j  =  1,  2,  •  •  •  ;  a  >  0,  and  is  the  Kronecker  delta. 

It  is  known  that  the  probabilities  in  terms  of  their  L.t.  71^(0.)  satisfy  the 
backward  Kolmogorov  system  of  equations  (see  Feller  [7]) 

(2.4)  (al  +  C)jt(a)  =  I  +  Cpn(a). 

If  the  solution  of  (2.4)  satisfies  an  (a)  1  =  lfora  >  0,  then  n  is  the  unique  solution 
of  (2.4)  and  is  also  the  unique  solution  of  the  forward  system  of  equations  given  by 

(2.5)  7t(a)(al  +  C)  =  I  +  ;r(a)Cp. 

Analogous  to  (2.4)  n  satisfies  the  backward  system 

(2.6)  (al  +  C  +  <5f)?i:(a)  =  I  +  Cp7r(a). 

In  [20],  it  was  shown  that  there  always  exists  a  solution  of  (2.6),  which  is  minimal 
among  all  its  solutions  and  which  also  is  the  minimal  solution  of  the  forward 
system,  analogue  of  (2.5) 

(2.7)  7i(a)(al  +  C  +  <5f)  =  I  +  jr(a)Cp. 

Let  Ij{t)  denote  the  indicator  function  of  the  set  [A (t)  =  f\.  Since 

(2.8)  Pitj(t)  =  E  ^exp  -j  —  S  Jy(X(T))dTj/;(*)  A(0)  = 

it  is  evident  that  knowledge  of  the  Pitj  is  equivalent  to  that  of  the  joint  distribu¬ 
tion  of  X{t)  and  Y{t).  With  this  in  mind,  in  [20]  the  problem  of  existence  and 
uniqueness  of  the  solution  of  (2.6)  and  (2.7)  was  studied  in  some  detail.  In 
particular,  if  the  chain  is  finite  with  SC  =  (1,2 ,  '  ’  •  ,  N},  N  <  oo,  then  it  can  be 
easily  seen  that  for  all  a  >  0,  the  matrix  (al  +  C  +  <5f  —  Cp)  has  an  inverse, 
so  that  from  (2.6)  and  (2.7)  we  have  the  explicit  solution  for  n(tx)  as 

(2.9)  ri(a)  =  (al  +  C  +  <5f  —  Cp)-1, 

valid  for  a  >  0  and  <5^0.  Let  =  (i jtx ,  >  '  *  *  >  *An)>  where 


(2.10)  ^(a)  =  exp  {  —  a/}  E\  exp  <  —  <5 


‘ /(*(*)) 
Jo 


dx  >  \X(0)  =  i)  dt. 


We  then  have 


(2.11) 


^(a)  =  n{a)l  =  (al  +  C  +  <5f  —  Cp)  1I. 


The  L.t.  i/q(a)  is  in  general  a  rational  function  of  a  and  can  therefore  be  easily 
inverted  to  yield  2?(exp  {  —  S  Jq/(A(t))  dx}  |X(0)  =  i). 
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Again  in  [20]  and  [21],  under  certain  assumptions,  we  proved  the  identity 

(2.12)  it  —  n  —  ditin , 

which  connects  n  and  n,  allowing  us  to  obtain  the  desired  n(oc)  in  terms  of  n(a), 
which  may  be  known.  The  identity  (2.12)  is  found  very  useful  in  applications 
particularly  because  of  the  manner  in  which  f  appears.  In  particular  in  [21],  we 
used  this  identity  to  obtain  the  joint  distribution  of  times  spent  by  the  Markov 
chain  in  various  states  of  a  given  finite  set  before  the  process  hits  a  given  taboo 
set. 

Remark.  A  formulation  alternative  to  the  consideration  of  the  M.c. 
{X(t),  Z(t)}  would  be  to  consider  a  modified  time  homogeneous  M.c.  X(t)  with 
state  space  { a ,  1,2,3,  *  *  *}  with  new  exponential  parameters,  say  cf,  given  by 

(2.13)  c,-  =  (Cj  +  Sfi)(  1  -  Sia).  i  =  a.  1,  2,  3.  •  •  •  , 

and  the  new  transition  matrix  ptj  given  by 


(2.14) 


Pij  =1 


CiPij(Ci  +  8 ft)  1 

<5/i(Ci  +  Sfi)~l 


for  i.j  =  1 ,  2,  •  •  •  , 
for  j  =  a,  i  =  1,2,*-*, 
for  i  =  a. 


so  that  the  state  a  is  an  absorption  state.  However,  since  for  each  formulation, 
the  relevant  information  concerning  the  distribution  of  X(t)  and  Y  (t)  is  contained 
in  the  equations  (2.6)  and  (2.7),  we  find  no  essential  gain  in  considering  this 
alternative  formulation. 

In  [20],  it  was  pointed  out  that  in  the  past  most  of  the  researchers  in  the  area 
touched  by  this  paper  have  exploited  the  backward  system  such  as  (2.6)  (see 
for  instance,  Gaver  [9],  Daley  [4],  Daley  and  Jacobs  [5],  and  McNeil  [12]). 
Forward  equations  (2.7)  were  not  used  possibly  because  of  lack  of  probabilistic 
interpretation.  The  present  method  via  the  quantal  response  process  Z(t)  has 
the  advantage  over  the  past  ones  in  that  it  provides  the  needed  probabilistic 
interpretation.  In  the  present  paper,  we  shall  exploit  the  forward  system  a  great 
deal,  by  applying  it  to  the  case  of  certain  well-known  processes  arising  in  several 
live  situations.  In  [20],  [21]  and  also  in  the  applications  of  the  method  exhibited 
in  the  present  paper,  we  have  restricted  outselves  mostly  to  Markov  chains  with 
countable  state  space.  However,  it  is  evident  that  the  method  is  applicable  to 
almost  all  types  of  continuous  time  stochastic  processes.  The  application  to 
certain  processes  such  as  semi-Markov  processes  will  be  dealt  with  elsewhere. 

Finally,  it  may  be  remarked  that  the  above  method  has  some  resemblance 
with  the  work  of  Kemperman  [11]  and  also  with  the  method  of  collective  marks 
due  to  van  Dantzig  [6] ;  in  the  present  case,  however,  the  approach  was  moti¬ 
vated  by  the  author’s  work  on  the  response  time  distribution  arising  in  certain 
biological  situations  (see  Puri  [16],  [18],  and  [19]). 
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3.  Birth  processes 

This  section  will  be  devoted  to  the  case  where  the  M.c.  X(t)  is  a  birth  process. 
Section  3.1  deals  with  the  time  homogeneous  case,  while  Section  3.2  deals  with 
linear  nonhomogeneous  birth  processes. 

3.1.  Time  homogeneous  birth  processes.  We  shall  use  here  the  notation  of 
Section  2.  Let  X(t)  be  a  time  homogeneous  birth  process  with  pjk  =  8j  +  1>k  and 
X  (0)  =  i.  Also  let 

(3.1)  N  =  min  {j ;  j  ^  i,  Cj  =  0}; 

if  Cj  >  0  for  all  j  ^  i,  then  N  =  go.  If  N  <  oc,  the  M.c.  is  a  finite  one  (with 
Cj  >  0 ,j  =  i,i  +  1,  •  •  •  ,  N  —  1,  and  cN  —  0),  a  case  which  was  already  con¬ 
sidered  in  Section  2  with  an  explicit  answer  given  by  (2.9).  However,  if  A  =  go, 
we  assume  that  2*=  ,•  cj~ 1  =  go  ,  so  that  with  probability  one  only  a  finite  number 
of  jumps  of  the  chain  are  allowed  in  any  finite  time  interval.  For  the  present  case, 
the  systems  of  equations  (2.6)  and  (2.7)  are  given  by 

(3.2)  (a  +  Ci  +  8fi)nik( a)  -  ci7ti+ 1>k(a)  =  Sik 
and 


(3.3)  (a  +  ck  +  8fk)nik(cc)  -  ck_^ ri.fc_1(a)  =  8ik. 

respectively,  with  7tfj-(a)  =  0  for  j  <  i  and  forj  >  N.  Each  of  these  systems  can 
be  uniquely  solved  for  nik( a)  recursively,  yielding 

{rt-  for  k  =  i, 

PiPi+i  '  ■  ■  Pk-irk  for  k  >  i. 

0  for  k  <  i  and  for  k  >  N, 


where  rj  =  (a  +  Cj  -f  Sfj)  1  and  pj  =  Cfj.  Because  of  the  condition  Cj  1  = 
oo,  it  is  now  easily  seen  that  for  a  >  0, 


(3.5) 


J  exp  {  —  a£}  E  ^exp  j  —  3  J  f(X(x))dz 


=  Z  =  lim  Z 

k=i  k=i 


X(0)  =  i)dt 


=  -  lim 

a  n-*N 


1  -  pipi+1  ■  •  •  pn  -  8  Z  fkPiPi  +  i  ■■■  Pk-i 

k  = 

N 

1  -  8  Z  fkPiPi+i  ■■■  Pk- irk 


where  if  A  <  oo ,  the  limit  of  a  sum  as  n  — *  N  is  taken  to  be  the  appropriate  finite 
sum.  Here  in  (3.5),  we  have  used  the  fact  that  a  2^=,-  uik(oc)  5=  1,  for  all  a  >  0. 
which  is  known  from  the  general  theory  of  Markov  chains.  The  fact  that 
lim„_*N  ( PiPi+x  '  '  '  Pn)  =  0  follows  from  the  condition  2^=i  cf  1  =  go. 
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Expressions  (3.4)  can  easily  be  inverted  to  yield  expressions  for  pik(t).  For 
instance,  if  Cj  +  Sfj  are  all  distinct  for  j  =  i,  i  +  1,  •  •  •  ,  A,  it  can  be  easily  shown 
that 


(k—1  \  .k  k  ~ 

n*  I  fl  (<V  +  bh  -  Cj  -  Sfj)-1  exp  { -  (Cj  +  Sfj)t}, 
j=  1  /  j  =  i  L(  =  U *j 


where  by  convention  (11*=/  Cj)  =  U^j  (c,  +  Sfe  —  Cj  —  Sfj)  1  =  1  for  k  = 
j  =  i.  Summing  (3.6)  over  k,  we  finally  obtain 


(3.7)  E  exp 


X(0)  =  * 


-  <5  f/(X(T)). 

=  z  ( n  cj]  z  [  n  (c*  +  bft  -  ci  -  exp ( -  (cj  +  <%)*}• 

k  —  i  \j  =  i  /  j  =  i  U  =  i,<*j  J 


Note  that  for  the  case  when  N  =  oo ,  an  interchange  of  the  two  summation  signs 
on  the  right  side  of  (3.7)  is  not  always  valid.  Also,  one  could  instead  invert  the 
L.t.  given  in  (3.5)  and  obtain  a  different  yet  equivalent  expression  for  (3.7). 

Again,  since  Y{t)  is  a  monotone  nondecreasing  function  of  f,  it  almost  surely 
converges  to  a  random  variable,  say  Y ,  as  t  — ►  oo .  By  using  a  Tauberian  argument 
it  follows  from  (3.5)  that 

i v 

(3.8)  ^(exp  {  — <5F}|X(0)  =  i)  =  lim  a  £  7tifc(a) 

a-0  fc=i 

=  1  -  lim  S  £  fkp*p*+i  •  *  •  Pk-iff, 

n-*N  k  =  i 


where  rf  =  (ct  +  Sfi)  1  and  pf  =  c,rf .  Now  after  some  manipulation  with  the 
right  side  of  (3.8),  it  can  be  shown  that 


(3.9) 


E(exp  {-5r}|X(0)  =  <)  =  lim  (pf  •  •  •  pf) 

n-»JV 


If  A  <  oo,  this  is  zero  (keeping  in  mind  that  cN  =  0)  unless  fN  =  0.  Thus,  if 
N  <  oo  and  fN  >  0,  Y  =  oo  a.s.  Again  if  A  =  oo,  (3.9)  is  equal  to  zero  if 
and  only  if  2]°=i  ( fj/cj )  =  oo,  in  which  case  also  Y  =  oo  a.s.  Let  Yf=i  (fj/cj)  < 
oo.  This  means  that  fN  =  0  whenever  A  <  oo.  Then  (3.9)  is  positive  for  all 
S  ^  0,  and  in  particular  is  equal  to  one  when  <5  =  0,  and  hence  Y  is  finite  a.s. 
Furthermore,  it  is  clear  from  (3.9)  that  Y  has  a  density.  In  fact,  if  we  assume  that 
fj/cj,  j  =  i,  i  +  1,  •  •  •  ,  A,  are  all  distinct,  then,  at  least  when  A  <  oo,  one  easily 
obtains  the  density  function  of  Y  as 


9(y)  = 


(3.10) 


y  >  o, 
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by  inverting  the  L.t.  (3.9).  As  expected,  it  follows  from  (3.9)  that  Y  can  be 
expressed  as  the  sum  Yfj=ifjXj,  where  rf,  ti  +  1,  •  •  •  ,  are  independently  negative 
exponentially  distributed  with  parameters  cf,  ci+1,  •  •  •  .  Here  Xj,j  =  i,i  +  1, 
•  •  •  ,  are  essentially  the  random  lengths  of  times  that  the  process  spends  in 
various  states. 

In  the  following  subsections,  we  consider  certain  special  cases  of  homogeneous 
birth  processes  that  arise  in  practice. 

3.1.1.  Case  of  a  simple  epidemic.  Jerwood  [10]  has  recently  considered  the 
case  of  a  simple  epidemic  which  starts  with  X(0)  =  i  infectives  and  S  (0)  =  N  —  i 
susceptibles  at  time  /  =  0,  where  N  <  oo.  If  X(t)  denotes  the  number  of  infec¬ 
tives  at  time  t,  then  X(t)  is  a  birth  process  as  considered  by  Bailey  [1]  with  the 
finite  state  space  (*,»  +  !,••*,  N),  N  being  the  absorption  state  and 


(3.11)  Cj  =  f$j(N  —  j),  j  =  i,i  +  1,  •  N. 

Jerwood  [10]  considers  the  distribution  of  the  cost  of  the  epidemic  exhibited  by 

(3.12)  Ci  =  aWt  +  bTt,  a  >  0,  b  >  0, 

where 

(3.13)  Wt  =  f  '  X(t)  dx,  T{  =  inf  { t :  X(t)  =  iV|A(0)  =  i}, 


a  is  the  cost  per  unit  time  per  infective,  and  b  is  the  cost  per  unit  time  both  over 
the  period  Tt.  Perhaps  a  more  realistic  situation  is  where  the  rate  of  the  first  cost 
varies  with  the  number  of  infectives  at  time  t,  so  that  one  would  like  to  obtain  the 
distribution  of 


(3.14) 
where 

(3.15) 


Ct  =  aW;  +  bTh  a  >  0,  b  >  0, 

w;  =  (Ti  h(x{x))dz, 

Jo 


with  0  ^  h(j)  <  oo,j  =  i,  i  -1-  1,  •  •  • ,  N  —  1.  Since  we  are  concerned  with  the 
epidemic  only  until  the  first  passage  time  Tt  to  state  N,  without  loss  of  generality, 
we  may  take  h (N)  =  0.  In  order  to  fit  this  into  our  situation  above,  all  we  need  to 
take  is 


(3.16) 


fj  = 


ah(j)  +  6, 

0, 


j  =  i,  i  4-  1,  •  •  •  ,  N  -  1, 
otherwise. 


Now  since  the  passage  to  the  absorption  state  N  occurs  with  probability  one,  it 
is  easy  to  see  that 


(3.17)  i£(exp  {  —  <5Cj})  =  lim  exp  <  —  <5 


f(X(r))dA\X{0)  =  i 


n-i  r  / 

=  E(exp  {-SY}\X(0)  =  i)  =  Y\  l+<5(^ 

j  =  i  L  \Cj/ 
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The  last  equality  follows  from  (3.9).  If  the  fj/cj  are  all  distinct,  then  the  density 
function  of  C{  is  given  by 


(3.18)  g(y)Hn-7^L 


N-  1 


j=Ai  fj\  i  \fr  fj 


exp 


cjy_ 
SjY 


y  >  o, 


the  expected  cost  being  ^(C,)  =  (fj/cj).  Incidentally  using  (3.16)  in  (3.17), 
we  have,  with  at  =  ad  and  a2  =  bS, 


N-  1 


(3.19)  £(exp  {-al  W[  +  a^})  =  [] 


1  + 


<*Mj)  +  a2 


oq  >  0,  a2  >  0, 


which,  by  treating  oq  and  oe2  as  the  dummy  variables,  is  the  joint  L.t.  of  the 
random  variables  IF/  and  Tt. 

3.1.2.  Poisson  process.  This  process  is  a  special  case  of  the  birth  process  of 
Section  3.1,  with  =  A  >  0  for  all  j.  In  this  case,  if  fj,j  =  i,  i  +  1,  •  •  •  ,  are 
all  distinct,  then  it  follows  from  (3.6)  that 


o 


(3.20)  E[  exp  <j  -  S  |  f(X{x))  dij  Ik(t) 

k 

exp  {-It}  £ 


X(0)  =  *  =  Pik(t) 


n  (//-A)-1 

Lt  =  iJ 


exp  {-dfjt}, 
k  =  i,  i  +  1 , 


A  special  case  with/,-  =  j  has  been  considered  elsewhere  by  the  author  (see  [18]). 
It  was  shown  there  that  for  Is  I  ^  1  and  <5^0  and  with  X  (0)  =  0, 


(3.21)  £'lsx<')exp 


X(x)  dx 


X(0)  =  0  )  =  exp 


A(  +  y(l  — 


With  s  =  1,  this  can  be  easily  inverted  to  yield  the  distribution  function  of  Y {t) 
given  by 


(3.22) 


Ht(y)  =  exp  {-A*}  £ 


(A0" 


F*(n)(y), 


n  =  o  ni 

where  F*(n)  stands  for  the  n  fold  convolution  of  Ft ,  the  distribution  function  of 
a  random  variable  uniformly  distributed  over  (0,  t). 

3.2.  Nonhomogeneous  birth  processes.  For  the  case  of  nonhomogeneous 
Markov  processes,  in  general,  one  finds  it  more  convenient  to  use  the  forward 
Kolmogorov  system  of  equations  than  the  backward.  Consider  a  birth  process 
A(£)  with  X(0)  =  i,  such  that  forj  =  i,  i  +  1,  *  •  •  , 

P(X(t  +  A  t)  =  j  +  l|X(f)  =  j)  =  Cj{t)Xt  +  o(At), 

P(X(t  +  At)  =  j\X{t)  =j)=  1  -  Cj{t)At  +  o{At). 

Also  for  the  function /of  the  integral  (1.1),  let 

(3.24)  f(j,t)  =  fj(t),  j  =  i,i  +  !,•••• 


(3.23) 
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Here  the  functions  Cj(t)  and  /•(/)  are  assumed  to  be  nonnegative,  continuous 
and  integrable  over  (0,  t)  for  every  finite  t.  Furthermore,  let  the  functions  Cj{t) 
be  such  that  the  process  X(t)  has,  with  probability  one,  a  finite  number  of  jumps 
in  any  finite  time  interval.  Then  using  (1.2)  and  (3.23),  one  obtains  in  a  standard 
manner  the  forward  differential  equations  for  the  probabilities  Pij(t)  as  given  by 

(3-25)  ~fft]  =  +  Cj-i(t)Pi,j-i(t),  j  =  i,i  +  I,'”, 


where  Pij(t)  =  0  for  all  j  <  i.  Recursively,  these  equations  can  be  solved  subject 
toi^-jO)  =  to  yield 


<Ci(t)  +  Sfi(T))dT: 


Pu(t)  =  exp  <  - 


(3.26) 
and 

(3.27)  P^t)  =  \  exp<j  —  |  (cj(u)  +  dfJ{u))du\cj-l(T)Pij-l{T)dT, 

j  =  i  +  1,  i  +  2,  •  •  •  . 


Unfortunately,  in  general,  there  appears  to  be  no  way  of  obtaining  the  expression 
for  Pij(t)  in  a  closed  form.  Instead,  in  the  rest  of  this  section,  we  restrict  ourselves 
to  the  case  of  linear  birth  processes  with 


(3.28) 


Cj(t)  =  a (t)  +  jv(t),  j  =  i,  i  +  1,  •  •  •  , 


and  with  fj(t)  =  Thus,  here  we  are  interested  in  the  integral  of  the  form 


(3.29) 


Y (t)  —  f  (3(t)X(t )  dz. 
Jo 


For  this  case,  the  equations  (3.25)  take  the  form 


(3.30) 


dPjj(t) 

dt 

=  _ [<*(o  +  j{dP(t)  +  v(t)Y\Pij(t)  +  [«(o  +  ( j  ~  i)v(0]^,j- 1 (0> 


for  j  =  i,  i  +  1 ,  •  •  •  .  Let 

00 

(3.31)  8{s,  t)  =  £  M  S  1. 

j  —  i 


Then  multiplying  both  sides  of  (3.30)  by  ,sJ  and  adding  overj  we  obtain,  after 
some  simplification,  the  equation  * 


Gt  —  [v(f)s  —  v(t)  —  S[l(t)~\sGs  =  —  oc(t)(\  —  s)G, 


(3.32) 
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where  here  and  elsewhere  Gt  and  Gs  denote  the  respective  partial  derivatives 
of  G.  Equation  (3.32)  is  subject  to  the  initial  condition  G(s,  0)  =  s'  and  can  be 
solved  by  standard  methods  yielding 


(3.33)  G (s,  t)  =  [>(«;  0,  *)]'  exp 
where,  for  0  ^  x  ^  t, 

(3.34)  [(f) (s ;  x ,  t)~\~ 1  =  -  exp 


I 


a(T) [i  —  T>  0]  dx  >, 


(v (u)  +  SP(u))  dwj 

J 

v(u)  exp  If  (v(v)  +  5f}(v))  dv\  du. 


Since  in  the  present  case  of  (3.28),  it  is  known  that  in  any  finite  time  interval, 
with  probability  one,  there  are  only  a  finite  number  of  jumps  of  the  process  X(t), 
(3.33)  with  s  =  1  yields 


(3.35) 


F^exp 


I 


—  S  /3(t)X(t)  dx 


X(0)  =  i 


G(l,t). 


In  the  next  subsection,  we  specialize  the  above  results  to  Polya  process  which 
arises  very  often  in  various  live  situations  such  as  the  theory  of  accident  prone¬ 
ness  (see  Bates  and  Neyman  [2]). 

3.2.1 .  Polya  process.  This  is  a  special  case  of  the  linear  birth  process  discussed 
above  with 

(3.36)  a(<)  =  X(l  +  pXt)~\  \{t)  =  Xp{\  +  pXt)~\  X  >  0,  p  >  0. 


Also,  we  consider  the  special  case  where  fi(t)  =  1,  so  that  we  are  interested  in 
the  integral  Y ( t )  =  X(t)  dx.  For  this  case,  expression  (3.33)  simplifies  to 


(3.37)  G(s,  t)  =  s'  exp  {—  iSt} 


(1  +  Xpt)  -  ^(1  -  e~dt)s 

o 


-(i+p  *) 


This  then  gives  the  joint  distribution  of  X  (t)  and  Y(t).  In  particular,  from  this  one 
easily  obtains,  for  n  =  0;  1,  2,  •  •  •  , 


(3.38) 


p,  ,„+((<)  =  (i  +  *p‘) 


~(i+P  '). 


■  idt  tL 


+  n  —  14- 


P  1 )  '  •  •  G  +  P  1 ) 
n\ 


(  Xpt  V  /l  -  e~*V 
\1  +  XptJ  y  St  J 
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Again,  one  can  easily  invert  (3.37)  with  s  =  1  to  yield  the  distribution  function 
Ht(y)  of  the  integral  Y(t),  given  by 

(3.39)  H,(y)  =  (1  + 

•  £  •  •  • (i  ±  »")(-?£ 'L\"  U  tF,<nHy) 

for  y  >  it,  and  for  y  ^  it,  Ht(y)  =  0.  Here  the  operation  *  denotes  the  convolu¬ 
tion  between  two  distribution  functions,  Ut  is  the  distribution  function  corres¬ 
ponding  to  a  degenerate  random  variable  taking  value  as  ( ti ),  and  F t*(B)  is  as 
defined  in  Section  3.1.2.  Finally,  if  we  let  i  =  0  and  p  -*■  0  in  (3.37),  we  obtain 
the  expression  (3.21)  for  the  Poisson  process  as  expected. 


4.  Birth  and  death  processes 

We  now  consider  the  case  of  a  time  homogeneous  birth  and  death  process 
(b-d  process)  X(t)  where,  in  notation  of  Section  2,  ck  =  (Xk  +  pk), 

(  +  Fk)~l  if. /  =  k  +  1, 

(4.1)  pkj  =  j  pk(Xk  +  pk)~l  iij  =  k  -  l, 

f  0  elsewhere, 

and  Ak  and  pk  are  nonnegative  constants  with  p0  =  0.  Let  X(0)  =  i.  The  back¬ 
ward  and  forward  equations,  analogues  of  (2.6)  and  (2.7)  but  converted  into 
differential  equations,  are  given  by 

(4.2)  =  -a,  +  +  sf,)P,kw  +  ,,»(<>  +  ftfl-i,»w 

at 

and 

(4.3)  —  =  ~(^k  Y  pk  +  dfk)Pik(t)  +  ^k- l^i,k-  1  (0  +  Pk+l^i.k  +  lit)’ 

at 

respectively.  Unfortunately,  we  shall  not  consider  these  here  in  this  generality. 
Instead,  we  shall  consider  certain  special  cases  which  arise  in  several  practical 
situations.  For  this,  we  shall  particularly  be  making  use  of  the  forward  system 

(4.3) .  Since  X(0)  =  i  will  be  kept  fixed,  we  shall  write  for  brevity  Pik(t)  =  Pk(t). 
4.1.  Linear  birth  and  death  processes  with  immigration.  Consider  a  b-d 

process  with  Xk  =  kX  4-  v  and  pk  =  kp  for  k  =  0,  1,  2,  •  *  * ,  where  X,  v,  and  p 
are  positive  constants.  Such  processes  arise  in  the  study  of  population  dynamics 
and  also  with  X  =  0  in  the  queueing  theory  of  M/M/co  queues.  Here  we  wish 
to  obtain  the  joint  distribution  of  X(t),  Jf0  X(t)  dx,  and  T(t),  the  last  one  being 
the  length  of  time  during  (0,  t)  the  process  remains  in  nonzero  states.  In  M/M/co 


492 


SIXTH  BERKELEY  SYMPOSIUM  :  PURI 


queue,  T(t)  represents  the  time  for  which  at  least  one  channel  remains  busy 
during  (0,  t) :  X  (x)  dx  represents  the  cumulative  time  the  customers  spend 
during  (0,  t)  while  they  are  being  served.  In  the  study  of  response  of  host  to 
injection  of  virulent  bacteria,  j(0  X(x)  dx  could  be  regarded  as  a  measure  of  the 
total  amount  of  toxins  produced  by  the  live  bacteria  during  (0,  t),  assuming  a 
constant  toxin  excretion  rate  per  bacterium  (see  Puri  [16],  [17],  and  [18]). 

In  order  to  accomplish  our  purpose,  it  is  sufficient  to  take 


(4.4) 
so  that 

(4.5) 


dfk  ~  Pl(^  o)  +  ^P  2, 


k  =  0,  1,2, 


Pk(t)  =  E  exp  <  -  <5  f(X(x))  dx\  Ik(t) 


=  E  ^exp  |  —  fiiT(t)  -  P2  J  ^(T) 


Furthermore,  for  the  present  ease,  the  system  (4.3)  takes  the  form 

(—  [v  +  Pi  +  k([i  +  A  +  P2)]Pk 

dPk{t)  J  +  (k  +  1  )fiPk  +  i  +  [v  +  (k  —  l)/i]Pfc_i-  k  ^  1, 
4.6  — - —  = 

dt  [  -  vP0  +  nPx .  k  =  0. 

Let  G(s,  t)  be  as  defined  in  (3.31).  Then  from  (4.6)  we  have 


(4.7)  Gt  —  /l(«s  —  rt)(s  —  r2)Gs  =  — [v(l  —  s)  +  PP]G  +  PiP0 


where  ry  and  r2  denote  with  plus  and  minus  signs,  respectively, 

(4.8)  —  [(/x  +  X  +  P2)  ±  {{n  +  X  +  P2)2  —  4 /U}1/2]. 

The  problem  now  is  to  solve  (4.7)  for  G  subject  to  the  side  condition  G(s,  0)  =  s'. 
This  can  be  accomplished  by  standard  methods.  We  give  here  only  the  final 
answer  in  terms  of  its  L.t.  Let 


(4.9)  I M«,  0  =  [M*.  0]‘[^ 2('G  0]  vMexp  {-(v  +  Pi  -  vr2)t} 
and 

(4.10)  \j/2{s,  t)  =  [, h2(s ,  /)]_vM  exp  {~(v  +  Pi  -  vr2)t }, 
where 

(4.11)  t)  =  {'*'  -  *>  +  till  -  '^)exP  z  IU£>\ 

l  (ri  ~  s)  +  (s  -  r2)  exp  {-X{ri  -  r2)t }  j 

and 

(4.12)  h2(s,  t )  =  {(*  -  r2 )  exp  {-/l  -  r2)t }  +  (rx  -  sjjfa  -  r2)~{ 


INTEGRAL  FUNCTIONALS 


493 


Also,  let  \f/*{s,  a),  i l/*(s,  a),  G*(s,  a),  and  P*  (a)  be  the  Laplace  transforms  over 
time  t  of  \jj2,  G,  and  P0,  respectively,  defined  for  a  >  0.  From  (4.7),  it  is 
easy  to  show  that 

(4.13)  G(s,  t)  =  t)  +  p i  f  P0(t  -  z)\l/2(s,  t)  dx, 

Jo 

from  which  it  follows  that 

(4.14)  G*(s,  a)  =  ^f(s,  a)  +  PiP*(^2 {s,  a). 

With  «  =  0,  (4.14)  yields 

(4.15)  P0*(«)  =  <W(0,  a)[l  -  (0,  a)]"1. 

Finally,  by  using  this  in  (4.14),  we  obtain 

(4.16)  G*{s,  a)  =  i a)  +  P^${s,  a)t/r?(0,  a)[l  -  ^^(0,  a)]-1. 

We  now  consider  briefly  a  special  case  with  no  immigration,  that  is,  with 
v  =  0,  where  we  explicitly  obtain 

(4.17)  P0(t)  =  Q  {exp  {-^1i}[/(0]i  +  Pi  J  exp  { -^1T}[/(t)]i  dr}, 
and 

(4.18)  <9(5,  t)  =  exp  {-0i<}  j/?!  j^exp  {p1x}P0{x)dx  +  [Aj(a,  f)]'|, 
with 

(4.19)  J{t)  =  [1  -  exp  {  —  A(rx  -  r2)t}']{rl  -  r2  exp  {-X{r1  -  r2)t]~\~x. 

The  case  without  the  random  variable  T(t)  has  previously  been  considered 
elsewhere  by  the  author  [15].  For  the  present  case  (v  =  0),  it  is  known  that 
P(X(t)  ->  0  or  oo)  =  1  and  that  P(X(t)  -*■  0)  =  min  (1,  n/X).  Also,  T(t)  being 
a  nondecreasing  function  of  t  tends  almost  surely  to  a  random  variable  T  as 
t  -*■  oo.  Here  T  is  the  first  passage  time  of  the  process  to  the  state  zero.  Also 
P(T  <  oo)  =  min  (1,  n/X).  Thus  by  using  (4.17),  we  have 


if  p!  =  0, 
if  Pi  >  0. 
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On  the  other  hand,  in  the  presence  of  immigration  (v  >  0),  if  we  wish  to  find 
the  joint  distribution  of  T  and  X(r)  dx,  we  first  modify  our  process  (see  also 
[21])  by  taking  A0  =  0,  Xk  =  kX  +  v  for  k  ^  1,  so  that  zero  is  an  absorption 
state  of  the  process.  Here  we  allow  immigration  only  as  long  as  the  process  has 
not  touched  the  state  zero.  The  analogue  of  equation  (4.7)  for  the  modified 
process  is  then  given  by 

(4.21)  a,  -  A(«  -  >•,)(«  -  ri)Ss  =  ~[fit  +  v(l  -  *)](<?  -  P0). 

The  solution  of  (4.21)  subject  to  G(s,  0)  =  s\  i  ^  1,  is  given  by  (4.13)  through 
(4.16)  with  \J/ 2  replaced  by 

(4.22)  i j/2(s,  t) 

1  , 

= 

Pi 

and  if/*  by  t^2  •  We  thus  have  from  the  new  (4.16),  while  using  aTauberian  argument, 


(v  +  Pi  ~  vrt)  + 


Hr i  -  r2)(r1  -  s) 


(r i  -s)  +  (s  -  r2)exp{-A(r1  -  r2)t}\ 


(4.23)  E  exp 


-Pi  T  ~  P2  |  X(t)  dx 

o 


X(0)  =  i 


=  lim  5(1,  t)  =  lim  a5*(l,  a) 

f-*  oo  a-*  0 


=  lim 

a-0 


#*(1,  a)  + 


«)jAT (0.  a) 
1  -  PyfoiO,  a)  _ 


On  the  other  hand,  it  can  be  easily  shown  that 


(4.24) 


lim  ai/f*(l,  a)  =  lim  t)  —  0, 

a-*0  f-*oo 

lim  i^f(l,  a)  =  f  \j/2(l,t)dt, 
a  -*  0  Jo 


and 

(4.25) 


lim  a^T(0,  1)[1  —  P ^5(0,  a)]-1 

=  lim  P0{t)  =  P0{ go) 

00 

f  exp  {  —  (v  +  Py  -  vr2)t}[hy(0,  t)]llh2(0,  t)yv/x  dt 
_  Jo _ 

f  exp  {  —  (v  +  Py  -  vr2)t}[h2{0,  <)]~vM  dt 
Jo 


Thus,  we  finally  have  from  (4.23), 

(4.26)  E  ^exp  j  —  T  —  /?2  f  X(x)dx 


X(0)  =i  =  PyP0(cc) 


$2(1,  t)  dt. 
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4.2.  M/M/I  queue.  This  corresponds  to  the  case  with  Xk  =  X  for  k  ^  0  and 
fik  ~  A*  f°r  k  =  1  with  /i0  =  0.  This  is  the  ease  which  has  recently  been  explored 
by  Gaver  [9],  Daley  [4],  Daley  and  Jacobs  [5],  and  McNeil  [12].  Most  of  these 
authors  have  used  the  backward  system  analogues,  while  we,  based  on  our 
method,  shall  use  the  forward  system.  Also,  this  section  will  apparently  have 
some  relevance  to  the  paper  presented  by  Professor  Gani  at  this  Symposium. 
In  the  case  of  M/M/I  queue,  T(t)  as  defined  in  the  previous  section  represents 
the  period  for  which  the  channel  remains  busy  during  (0,  t)  and  [JrQ  X(t)  dt  — 
T(t) ]  represents  the  total  time  wasted  by  the  customers  during  (0,  t)  while 
standing  in  the  queue  and  waiting  for  their  turn  for  service.  Although  these 
random  variables  are  of  some  practical  importance,  to  the  best  of  author’s 
knowledge,  their  distributions  have  not  been  considered  before.  The  integrals 
studied  by  Gaver,  Daley,  and  others  were  restricted  only  to  a  busy  period  of 
the  queue,  where  “zero”  acts  as  an  absorption  state.  We  shall  touch  this  case 
briefly  later. 

As  before,  we  are  interested  in  obtaining  the  joint  distribution  of  X(t), 
jt0X(t)dr,  and  T(t).  The  analogue  of  equation  (4.7)  for  the  present  case  is 
given  by 

(4.27)  sGs  +  fl2s2Gs  =  A(*  -  fj)(«  -  r2)G  +  [(fi  +  -  fi\P0, 

which  is  to  be  solved  subject  to  G(s,  0)  =  sl.  Here  r1  and  r2,  with  positive  and 
negative  signs,  respectively,  are  given  by 

(4.28)  ±l(n  +  X  +  jS.)  ±{U>  +  X  +  /?,)2  -  4#U}''2]. 

Unfortunately,  the  solution  of  (4.27)  appears  quite  complex  and  involves  Bessel 
functions.  The  author  did  not  succeed  in  obtaining  an  explicit  solution  of  (4.27). 
However,  one  can  easily  solve  it  when  jS2  =  9,  giving  only  the  joint  distribution 
of  X(t)  and  T(t).  Taking  L.t.  of  (4.27)  (with  fi2  =  0)  with  respect  to  t,  we  obtain 
for  a  >  0, 

(4.29)  <?*(«,  a)  =  O  -  (ft  +  P,)s}Pg( «)  -  «'+1][A(s  -  rf )(«  -  r?)]"1, 
where  rj  and  r2,  with  positive  and  negative  signs,  respectively,  are 

(4-3°)  ^  [(/*  +  A  +  jSj  +  a)  ±  {{n  +  X  +  p1  +  a)2  -  4/iA}1/2], 

and  they  satisfy  the  relation  0  <  r%  <  1  <  rj.  Since  (5*  is  analytic  for 
M  <  1,  the  first  of  the  two  expressions  on  the  right  side  of  (4.29)  must  vanish 
at  s  =  r*.  This  fact  yields 


(4.31) 


A*  (*)  =  (^2  )i  + 1  [M  -  (M  +  £i)r*]  K 
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On  substitution  of  this  in  (4.29),  we  obtain 


(4.32)  G*(s.  a) 


(rf)i+1[/x  -  (fi  +  fi^s]  -  si+1[n  -  (H  +  gijrg] 
A(*  -  r*){s  -  rf)[/i  -  {H  +  /?i)rf ] 


Finally,  on  putting  s  =  1  in  (4.32)  and  inverting  the  resultant  L.t.  by  lengthy 
yet  standard  methods  (see  Saaty  [22]),  we  obtain 

00 

(4.33)  <9(1,  t)  =  E(e\ p  {  -fit  T(t)}  |X(0)  =  i)  =  £  ^.(0. 

n  =  0 


where 


(4.34)  /*,(<) 

=  exp  { —  (A  +  /i  +  /?x)/} 

/i  +  /?1//A(I'-"-1)/2 


(/  —  «)/ 2 


+ 


U 

+  IV 


A 

(i-i)/2 


^n  +  i  +  1  (£) 


(1  -  A/i(/i  +  jSj)  2)  X  [(/i  +  £i)(A/i)  1/2]fc£„  +  i+k(£)>, 


and  A?„(w)  denotes  the  Bessel  function  of  the  first  kind  and  ^  =  2 (A/i)1/2L 
We  now  consider  only  the  busy  period  of  the  M/M/I  queue  started  with 
X(0)  =  i.  For  this  we  take  Ak  =  A  for  k  ^  1,  and  A0  =  0.  so  that  “zero”  is  an 
absorption  state  of  the  process  X(t).  As  before  T(t)]T  a.s.  as  t  — >  oo,  where  T 
is  the  length  of  the  busy  period.  If  /i  ^  A,  it  is  known  that  X (/)-»•  0  with 
probability  one,  so  that  P(T  <  oo)  =  1 .  On  the  other  hand  if  fi  <  X,P{T  <  oo)  = 
P(X(t)  ->■  0)  =  fi/X,  and  P(T  =  oo)  =  P{Mt)  -*■  oo)  =  1  —  (/i/A).  The  ana¬ 
logue  of  (4.27)  now  takes  the  form 

(4.35)  sGt  +  (32$2GS  =  [As2  —  (A  +  /j.  +  Pi)s  +  /i](U  —  AJ0), 


to  be  solved  subject  to  G(s ,  0)  =  s'.  Unfortunately,  the  solution  of  this  presents 
similar  difficulties  as  of  equation  (4.27).  However,  once  it  is  solved  we  have  the 
desired  result  given  by 


(4.36)  lim  G(  1,  t)  =  E  \  exp  <  —  ^  T  —  /?2  X(x)  dx 


X(0)  =  i 


This  result  has  been  studied  through  other  methods  by  Daley  [4]  and  Daley  and 
Jacobs  [5].  Again,  the  equation  (4.35)  can  be  easily  solved  like  (4.27)  when 
fi 2  =  0.  However,  since  the  distribution  of  the  length  of  the  busy  period  T  is 
already  known  (see  Saaty  [22]),  we  shall  not  pursue  this  further  here. 
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5.  Illness-death  processes 

These  processes  have  been  extensively  studied  by  Fix  and  Neyman  [8]  and 
more  recently  by  Chiang  [3],  Briefly,  these  are  finite  Markov  chains  with  iwo 
sets  of  states;  Sh  i  =  1,  2,  •  •  •  ,  s,  are  the  illness  states  and  Re.  0  =  1,  2,  •  •  •  ,  r, 
are  the  death  states.  (In  this  section,  symbols  i,j ,  and  k  will  stand  for  S{,  Sj.  and 
Sk,  and  8  for  Rd.)  Here  all  the  death  states  are  absorption  states.  Also,  in  terms 
of  the  notation  of  Section  2,  for  the  transitions  among  the  various  states  we  have 
on  adopting  Chiang’s  notation, 


ciPij 

(5.1)  Cipie 

—  C: 


vij- 

*  +  j-  =  1  2.  •  • 

•  .  <s 

kid- 

i  =  1.  2.  •  •  •  .  *,  8  =  1. 2.  •  • 

•  .  r 

Vii- 

i  =  1. 2.  •  • 

‘  r  8 

Consider  a  typical  person  moving  from  one  state  to  another  according  to  the 
above  M.c.  until  he  is  absorbed  into  one  of  the  death  states.  Chiang  ([3], 
pp.  81  and  160)  has  considered  the  lengths  of  this  person's  stay  in  various  states 
within  a  period  of  length  t,  and  has  given  expressions  for  their  expected  values 
only;  while  our  method  leads  easily  to  their  joint  distribution.  For  this,  take 

„  „  f<3.-  if  /  is  the  state  S{. 

(5.2)  Sf,  =  < 

if  /  is  the  state  Re. 


Analogous  to  Chiang’s  notation  (see  [3],  p.  152).  let 


(5.3)  Pij(t)  =  P(Z(t)  =  1.  X(t)  =  Sj\X(0)  =  S(.  Z(0)  =  1) 
and 

(5.4)  Qw(t)  =  P{Z(t)  =  1,  X(t)  =  Re\X(0)  =  Si,  Z( 0)  =  l). 


where  Z{t)  represents  the  “quantal  response  process"  as  defined  in  Section  1. 
Then  the  forward  system  of  equations  for  P  are  given  by 

(5.5)  ~'Jj  =  -(X  vjk  +  X  kjO  +  Si)Pij{t)  +  X  Pkit)Vkj 

k*j  6  k±j 

for  i,j  =  1,  2,  •  •  ,  s.  Similarly,  one  could  write  down  the  backward  system. 
Either  of  these  systems  can  be  uniquely  solved  for  Pij{t  ).  which  then  can  be  used 
to  obtain  Qie{t)  by  noticing  that 

(5.6)  Qie{t)  =  £  jxje  Pij{T)  exp  {-a e(t  -  t)}  dx,  8  =  1 ,  2,  •  •  •  ,  r. 

j=  i  Jo 
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Here  the  factor  exp  {  —  oe(t  —  t)}  under  the  integral  sign  denotes  the  probability 
that  our  hypothetical  animal  of  the  “quantal  response  process”  Z(t)  does  not 
die  during  (t,  t),  once  the  process  has  touched  the  state  R0  at  moment  t.  Let 
X(0)  =  Sh  and 

Tij(t )  =  length  of  stay  in  Sj  during  (0,  t),  j  =  1,  2,  •  •  •  ,  s, 
1  Ti0(t)  =  length  of  stay  in  Re  during  (0 ,  t),  6  =  1,  2,  •  •  •  ,  r. 

Having  obtained  P [j  and  Qi0  in  the  above  manner,  the  L.t.  of  the  joint  distribu¬ 
tion  of  Tij(t)  and  Ti0(t)  is  then  given  by 

exp  j-  X  sJTiA o  -  Z  *A(*)})  =  Z  Fiji*)  +  Z  Qio(t), 

l  J=  i  «  =  i  )/  J= i 

where  Sj  and  a0  act  as  the  dummy  variables  for  the  L.t.  The  L.t.  (5.8),  in 
general,  is  a  rational  function  of  <5’s  and  <r’s  and  can  be  inverted  by  standard 
methods  to  yield  the  desired  distribution.  We  shall  illustrate  the  above  approach 
through  an  example,  where  we  take  s  =  2,  r  =  1,  and  X(0)  =  Si .  Since 


(5.9)  Tu(t)  +  Ti2{t)  +  Tn(t)  =  t  a.s., 


it  is  sufficient  to  study  the  joint  distribution  of  Tu(t )  and  Ti2(t)  only,  in  which 
case  we  may  take  ax  =  0.  The  equations  (5.5)  are  now  given  by 


(5.10) 

Let 


=  —  (V12  +  Pll  +  ^l)Al  +  V21^12> 

dP 

— H  =  — (v21  4-  H2i  +  $2)? 12  +  v12Pn. 


(5.11)  A  =  (S i  +  v12  4-  n n),  B  —  (S2  +  v21  -f  n2i), 

and  a  1  and  a2,  with  positive  and  negative  signs,  respectively,  be  given  by 

(5.12)  i[-(^  +  B)  ±  {(4  -  B)2  +  4v12v21}1/2]. 

The  solution  of  (5.10)  is  given  by 

Pn(t)  =  [(04  4-  B)  exp  {a^}  -  (B  +  a2)  exp  {a20](ai  _  a2)_1> 

(5.13)  ~ 

Pi2(0  =  v12[exp  {M}  ~  exp  |a2*}J(ai  -  «2)  • 

Now  using  (5.6)  with  a0  =  0,  we  can  obtain  Quit).  Finally,  omitting  details, 
we  have  by  using  (5.8), 

(5.14)  l£(exp  {—diTu  (t)  -  <527\2(*)}) 

=  P11  (t)  +  P\2(t )  4-  Qn{t) 

=  {[(«!  +  B  +  v12  4-  flu)  +  ( H11B  +  H21V1 2)ai_1]  exp  {<M} 

—  [(5  +  a2  4-  v12  +  Hu)  +  (HuB  +  H-2 1  vi 2 )a2  *] 

•exp  {a2t}j(ai  -  a2)_1  +  {(P11 B  4-  v12//21  )}(AB  -  v12v21)_1. 
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Since  Tll(t)  and  T12(t)  are  monotone  nondecreasing  functions  of  /,  Txl(t)]  Tlx 
and  Tl2(t)]T12  almost  surely  as  t  — ►  oo .  Here  the  random  variables  Tx  x  and  Tx  2 
represent  the  lengths  of  time  the  person  spends  in  Sx  and  S2 ,  respectively,  before 
he  finally  dies.  Letting  t  — ►  oo  and  using  the  fact  that  ax  and  a2  are  negative,  we 
obtain  from  (5.14) 


(5.15)  2£(exp  {  —  STX  x 


s2tX2}) 


^ll(V2l  +  /*21  +  jr)  +  V12^21 
(Sx  +  V12  +  ^li)(^2  +  V2 1  +  ^2l)  —  V 1  2  V2 1 


One  can  easily  invert  this  transform  to  give  explicitly  the  joint  distribution  of 
Txx  and  Tx2.  However,  we  shall  not  venture  into  this  here.  Instead,  we  refer  the 
reader  to  [21]  for  further  details  concerning  the  sojourn  times  of  the  type 
Txx,  Ti2,  and  so  forth,  and  close  with  the  remark  that  Tx  j  and  Tl2,  in  the  present 
case,  are  positively  correlated.  Furthermore,  marginally  each  one  of  them  is 
(negative)  exponentially  distributed  with  a  positive  probability  mass  at  zero 
only  in  the  case  of  Tl2. 
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